What happened about the reproducible builds effort for this week: Media coverage Debian's effort on reproducible builds has been covered in the June 2015 issue of Linux Magazin in Germany. Toolchain fixes

• gregor herrmann uploaded libextutils-depends-perl/0.404-1 which makes its output deterministic.
• Christian Hofstaedtler uploaded yard/0.8.7.4-2 which will not write timestamps in the generated documentation. Original patch by Chris Lamb, does not write timestamps in the generated documentation anymore.
• Emmanuel Bourg uploaded maven-plugin-tools/3.3-2 which removes the date from the plugin descriptor. Patch by Reiner Herrmann.
• Emmanuel Bourg uploaded maven-archiver/2.6-1 which now uses the date set in the DEB_CHANGELOG_DATETIME environment variable for the timestamp in the pom.properties file embedded in the jar files. Original patch by Chris West.
• Nicolas Boulenguez uploaded dh-ada-library/6.4 which will warn against non deterministic ALI for sources newer than changelog.

josch rebased the experimental version of debhelper on 9.20150507. Packages fixed The following 515 packages became reproducible due to changes of their build dependencies: airport-utils, airspy-host, all-in-one-sidebar, ampache, aptfs, arpack, asciio, aspell-kk, asused, balance, batmand, binutils-avr, bioperl, bpm-tools, c2050, cakephp-instaweb, carton, cbp2make, checkbot, checksecurity, chemeq, chronicle, cube2-data, cucumber, darkstat, debci, desktop-file-utils, dh-linktree, django-pagination, dosbox, eekboek, emboss-explorer, encfs, exabgp, fbasics, fife, fonts-lexi-saebom, gdnsd, glances, gnome-clocks, gunicorn, haproxy, haskell-aws, haskell-base-unicode-symbols, haskell-base64-bytestring, haskell-basic-prelude, haskell-binary-shared, haskell-binary, haskell-bitarray, haskell-bool-extras, haskell-boolean, haskell-boomerang, haskell-bytestring-lexing, haskell-bytestring-mmap, haskell-config-value, haskell-mueval, haskell-tasty-kat, itk3, jnr-constants, jshon, kalternatives, kdepim-runtime, kdevplatform, kwalletcli, lemonldap-ng, libalgorithm-combinatorics-perl, libalgorithm-diff-xs-perl, libany-uri-escape-perl, libanyevent-http-scopedclient-perl, libanyevent-perl, libanyevent-processor-perl, libapache-session-wrapper-perl, libapache-sessionx-perl, libapp-options-perl, libarch-perl, libarchive-peek-perl, libaudio-flac-header-perl, libaudio-wav-perl, libaudio-wma-perl, libauth-yubikey-decrypter-perl, libauthen-krb5-simple-perl, libauthen-simple-perl, libautobox-dump-perl, libb-keywords-perl, libbarcode-code128-perl, libbio-das-lite-perl, libbio-mage-perl, libbrowser-open-perl, libbusiness-creditcard-perl, libbusiness-edifact-interchange-perl, libbusiness-isbn-data-perl, libbusiness-tax-vat-validation-perl, libcache-historical-perl, libcache-memcached-perl, libcairo-gobject-perl, libcarp-always-perl, libcarp-fix-1-25-perl, libcatalyst-action-serialize-data-serializer-perl, libcatalyst-controller-formbuilder-perl, libcatalyst-dispatchtype-regex-perl, libcatalyst-plugin-authentication-perl, libcatalyst-plugin-authorization-acl-perl, libcatalyst-plugin-session-store-cache-perl, libcatalyst-plugin-session-store-fastmmap-perl, libcatalyst-plugin-static-simple-perl, libcatalyst-view-gd-perl, libcgi-application-dispatch-perl, libcgi-application-plugin-authentication-perl, libcgi-application-plugin-logdispatch-perl, libcgi-application-plugin-session-perl, libcgi-application-server-perl, libcgi-compile-perl, libcgi-xmlform-perl, libclass-accessor-classy-perl, libclass-accessor-lvalue-perl, libclass-accessor-perl, libclass-c3-adopt-next-perl, libclass-dbi-plugin-type-perl, libclass-field-perl, libclass-handle-perl, libclass-load-perl, libclass-ooorno-perl, libclass-prototyped-perl, libclass-returnvalue-perl, libclass-singleton-perl, libclass-std-fast-perl, libclone-perl, libconfig-auto-perl, libconfig-jfdi-perl, libconfig-simple-perl, libconvert-basen-perl, libconvert-ber-perl, libcpan-checksums-perl, libcpanplus-dist-build-perl, libcriticism-perl, libcrypt-cracklib-perl, libcrypt-dh-gmp-perl, libcrypt-mysql-perl, libcrypt-passwdmd5-perl, libcrypt-simple-perl, libcss-packer-perl, libcss-tiny-perl, libcurses-widgets-perl, libdaemon-control-perl, libdancer-plugin-database-perl, libdancer-session-cookie-perl, libdancer2-plugin-database-perl, libdata-format-html-perl, libdata-uuid-libuuid-perl, libdata-validate-domain-perl, libdate-jd-perl, libdate-simple-perl, libdatetime-astro-sunrise-perl, libdatetime-event-cron-perl, libdatetime-format-dbi-perl, libdatetime-format-epoch-perl, libdatetime-format-mail-perl, libdatetime-tiny-perl, libdatrie, libdb-file-lock-perl, libdbd-firebird-perl, libdbix-abstract-perl, libdbix-class-datetime-epoch-perl, libdbix-class-dynamicdefault-perl, libdbix-class-introspectablem2m-perl, libdbix-class-timestamp-perl, libdbix-connector-perl, libdbix-oo-perl, libdbix-searchbuilder-perl, libdbix-xml-rdb-perl, libdevel-stacktrace-ashtml-perl, libdigest-hmac-perl, libdist-zilla-plugin-emailnotify-perl, libemail-date-format-perl, libemail-mime-perl, libemail-received-perl, libemail-sender-perl, libemail-simple-perl, libencode-detect-perl, libexporter-tidy-perl, libextutils-cchecker-perl, libextutils-installpaths-perl, libextutils-libbuilder-perl, libextutils-makemaker-cpanfile-perl, libextutils-typemap-perl, libfile-counterfile-perl, libfile-pushd-perl, libfile-read-perl, libfile-touch-perl, libfile-type-perl, libfinance-bank-ie-permanenttsb-perl, libfont-freetype-perl, libfrontier-rpc-perl, libgd-securityimage-perl, libgeo-coordinates-utm-perl, libgit-pureperl-perl, libgnome2-canvas-perl, libgnome2-wnck-perl, libgraph-readwrite-perl, libgraphics-colornames-www-perl, libgssapi-perl, libgtk2-appindicator-perl, libgtk2-gladexml-simple-perl, libgtk2-notify-perl, libhash-asobject-perl, libhash-moreutils-perl, libhtml-calendarmonthsimple-perl, libhtml-display-perl, libhtml-fillinform-perl, libhtml-form-perl, libhtml-formhandler-model-dbic-perl, libhtml-html5-entities-perl, libhtml-linkextractor-perl, libhtml-tableextract-perl, libhtml-widget-perl, libhtml-widgets-selectlayers-perl, libhtml-wikiconverter-mediawiki-perl, libhttp-async-perl, libhttp-body-perl, libhttp-date-perl, libimage-imlib2-perl, libimdb-film-perl, libimport-into-perl, libindirect-perl, libio-bufferedselect-perl, libio-compress-lzma-perl, libio-compress-perl, libio-handle-util-perl, libio-interface-perl, libio-multiplex-perl, libio-socket-inet6-perl, libipc-system-simple-perl, libiptables-chainmgr-perl, libjoda-time-java, libjsr305-java, libkiokudb-perl, liblemonldap-ng-cli-perl, liblexical-var-perl, liblingua-en-fathom-perl, liblinux-dvb-perl, liblocales-perl, liblog-dispatch-configurator-any-perl, liblog-log4perl-perl, liblog-report-lexicon-perl, liblwp-mediatypes-perl, liblwp-protocol-https-perl, liblwpx-paranoidagent-perl, libmail-sendeasy-perl, libmarc-xml-perl, libmason-plugin-routersimple-perl, libmasonx-processdir-perl, libmath-base85-perl, libmath-basecalc-perl, libmath-basecnv-perl, libmath-bigint-perl, libmath-convexhull-perl, libmath-gmp-perl, libmath-gradient-perl, libmath-random-isaac-perl, libmath-random-oo-perl, libmath-random-tt800-perl, libmath-tamuanova-perl, libmemoize-expirelru-perl, libmemoize-memcached-perl, libmime-base32-perl, libmime-lite-tt-perl, libmixin-extrafields-param-perl, libmock-quick-perl, libmodule-cpanfile-perl, libmodule-load-conditional-perl, libmodule-starter-pbp-perl, libmodule-util-perl, libmodule-versions-report-perl, libmongodbx-class-perl, libmoo-perl, libmoosex-app-cmd-perl, libmoosex-attributehelpers-perl, libmoosex-blessed-reconstruct-perl, libmoosex-insideout-perl, libmoosex-relatedclassroles-perl, libmoosex-role-timer-perl, libmoosex-role-withoverloading-perl, libmoosex-storage-perl, libmoosex-types-common-perl, libmoosex-types-uri-perl, libmoox-singleton-perl, libmoox-types-mooselike-numeric-perl, libmousex-foreign-perl, libmp3-tag-perl, libmysql-diff-perl, libnamespace-clean-perl, libnet-bonjour-perl, libnet-cli-interact-perl, libnet-daap-dmap-perl, libnet-dbus-glib-perl, libnet-dns-perl, libnet-frame-perl, libnet-google-authsub-perl, libnet-https-any-perl, libnet-https-nb-perl, libnet-idn-encode-perl, libnet-idn-nameprep-perl, libnet-imap-client-perl, libnet-irc-perl, libnet-mac-vendor-perl, libnet-openid-server-perl, libnet-smtp-ssl-perl, libnet-smtp-tls-perl, libnet-smtpauth-perl, libnet-snpp-perl, libnet-sslglue-perl, libnet-telnet-perl, libnhgri-blastall-perl, libnumber-range-perl, libobject-signature-perl, libogg-vorbis-header-pureperl-perl, libopenoffice-oodoc-perl, libparse-cpan-packages-perl, libparse-debian-packages-perl, libparse-fixedlength-perl, libparse-syslog-perl, libparse-win32registry-perl, libpdf-create-perl, libpdf-report-perl, libperl-destruct-level-perl, libperl-metrics-simple-perl, libperl-minimumversion-perl, libperl6-slurp-perl, libpgobject-simple-perl, libplack-middleware-fixmissingbodyinredirect-perl, libplack-test-externalserver-perl, libplucene-perl, libpod-tests-perl, libpoe-component-client-ping-perl, libpoe-component-jabber-perl, libpoe-component-resolver-perl, libpoe-component-server-soap-perl, libpoe-component-syndicator-perl, libposix-strftime-compiler-perl, libposix-strptime-perl, libpostscript-simple-perl, libproc-processtable-perl, libprotocol-osc-perl, librcs-perl, libreadonly-xs-perl, libreturn-multilevel-perl, librivescript-perl, librouter-simple-perl, librrd-simple-perl, libsafe-isa-perl, libscope-guard-perl, libsemver-perl, libset-tiny-perl, libsharyanto-file-util-perl, libshell-command-perl, libsnmp-info-perl, libsoap-lite-perl, libstat-lsmode-perl, libstatistics-online-perl, libstring-compare-constanttime-perl, libstring-format-perl, libstring-toidentifier-en-perl, libstring-tt-perl, libsub-recursive-perl, libsvg-tt-graph-perl, libsvn-notify-perl, libswish-api-common-perl, libtap-formatter-junit-perl, libtap-harness-archive-perl, libtemplate-plugin-number-format-perl, libtemplate-plugin-yaml-perl, libtemplate-tiny-perl, libtenjin-perl, libterm-visual-perl, libtest-block-perl, libtest-carp-perl, libtest-classapi-perl, libtest-cmd-perl, libtest-consistentversion-perl, libtest-data-perl, libtest-databaserow-perl, libtest-differences-perl, libtest-file-sharedir-perl, libtest-hasversion-perl, libtest-kwalitee-perl, libtest-lectrotest-perl, libtest-module-used-perl, libtest-object-perl, libtest-perl-critic-perl, libtest-pod-coverage-perl, libtest-script-perl, libtest-script-run-perl, libtest-spelling-perl, libtest-strict-perl, libtest-synopsis-perl, libtest-trap-perl, libtest-unit-perl, libtest-utf8-perl, libtest-without-module-perl, libtest-www-selenium-perl, libtest-xml-simple-perl, libtest-yaml-perl, libtex-encode-perl, libtext-bibtex-perl, libtext-csv-encoded-perl, libtext-csv-perl, libtext-dhcpleases-perl, libtext-diff-perl, libtext-quoted-perl, libtext-trac-perl, libtext-vfile-asdata-perl, libthai, libthread-conveyor-perl, libthread-sigmask-perl, libtie-cphash-perl, libtie-ical-perl, libtime-stopwatch-perl, libtk-dirselect-perl, libtk-pod-perl, libtorrent, libturpial, libunicode-japanese-perl, libunicode-maputf8-perl, libunicode-stringprep-perl, libuniversal-isa-perl, libuniversal-moniker-perl, liburi-encode-perl, libvi-quickfix-perl, libvideo-capture-v4l-perl, libvideo-fourcc-info-perl, libwiki-toolkit-plugin-rss-reader-perl, libwww-mechanize-formfiller-perl, libwww-mechanize-gzip-perl, libwww-mechanize-perl, libwww-opensearch-perl, libx11-freedesktop-desktopentry-perl, libxc, libxml-dtdparser-perl, libxml-easy-perl, libxml-handler-trees-perl, libxml-libxml-iterator-perl, libxml-libxslt-perl, libxml-rss-perl, libxml-validator-schema-perl, libxml-xpathengine-perl, libxml-xql-perl, llvm-py, madbomber, makefs, mdpress, media-player-info, meta-kde-telepathy, metamonger, mmm-mode, mupen64plus-audio-sdl, mupen64plus-rsp-hle, mupen64plus-ui-console, mupen64plus-video-z64, mussort, newpid, node-formidable, node-github-url-from-git, node-transformers, nsnake, odin, otcl, parsley, pax, pcsc-perl, pd-purepd, pen, prank, proj, proot, puppet-module-puppetlabs-postgresql, python-async, python-pysnmp4, qrencode, r-bioc-graph, r-bioc-hypergraph, r-bioc-iranges, r-bioc-xvector, r-cran-pscl, rbenv, rlinetd, rs, ruby-ascii85, ruby-cutest, ruby-ejs, ruby-factory-girl, ruby-hdfeos5, ruby-kpeg, ruby-libxml, ruby-password, ruby-zip-zip, sdl-sound1.2, stterm, systemd, taktuk, tcc, tryton-modules-account-invoice, ttf-summersby, tupi, tuxpuck, unknown-horizons, unsafe-mock, vcheck, versiontools, vim-addon-manager, vlfeat, vsearch, xacobeo, xen-tools, yubikey-personalization-gui, yubikey-personalization. The following packages became reproducible after getting fixed:

• cwirc/2.0.0-8 uploaded by Colin Tuckley, original patch by Reiner Herrmann.
• darkplaces/0~20140513+svn12208-1 by Simon McVittie.
• exactimage/0.9.1-4 by Sven Eckelmann.
• gnupg/1.4.19-1 by Daniel Kahn Gillmor.
• httpunit/1.7+dfsg-11 by Emmanuel Bourg.
• hy/0.10.1-2 uploaded by Tianon Gravi, original patch by Reiner Herrmann.
• ioquake3/1.36+u20150412+dfsg1-2 by Simon McVittie, original patch by Reiner Herrmann.
• kiwi/1.9.22-3 by Jelmer Vernooij.
• lava-server/2015.05-1 uploaded by Neil Williams, original patch by Reiner Herrmann.
• libelixirfm-perl/1.1.976-4 uploaded by gregor herrmann, original patch by Chris Lamb.
• littler/0.2.3-2 by Dirk Eddelbuettel.
• mednafen/0.9.38.1-1 by Stephen Kitt.
• nftables/0.4-4 by Arturo Borrero Gonzalez.
• ntdb/1.0-7 by Jelmer Vernooij.
• onioncat/0.2.2+svn566-1 by intrigeri.
• openarena/0.8.8-13 by Simon McVittie.
• openarena-085-data/0.8.5split-6 by Simon McVittie.
• openarena-088-data/0.8.8-3 by Simon McVittie.
• openarena-data/0.8.5split-6 by Simon McVittie.
• openarena-maps/0.8.5split-6 by Simon McVittie.
• openarena-players/0.8.5split-6 by Simon McVittie.
• openarena-players-mature/0.8.5split-6 by Simon McVittie.
• openarena-textures/0.8.5split-6 by Simon McVittie.
• pybik/2.0-1 by B. Clausius.
• python-xmp-toolkit/2.0.1+git20140309.5437b0a-1 by Daniel Stender.
• quakespasm/0.90.0-3 by Stephen Kitt.
• traceroute/1:2.0.21-1 uploaded by Laszlo Boszormenyi, original patch by Lunar.
• unar/1.8.1-4 uploaded by Matt Kraai, original patch by Lunar.
• websvn/2.3.3-1.3 uploaded by Thijs Kinkhorst, original patch by Chris Lamb.
• xd/3.23.01-2 uploaded by Frank B. Brokken, original patch by Chris Lamb.

Some uploads fixed some reproducibility issues but not all of them:

• ada-reference-manual/1:2012.2-5 by Nicolas Boulenguez.
• apparmor/2.9.2-2 by intrigeri.
• argyll/1.7.0+repack-1 by J rg Frings-F rst.
• lava-dispatcher/2015.05-1 by Neil Williams.
• libaunit/3.7.1-2 by Nicolas Boulenguez.
• libflorist/2014-2 by Nicolas Boulenguez.
• mailcrypt/3.5.9-8 uploaded by Barak A. Pearlmutter, original patch by Chris Lamb.
• openchange/1:2.2-7 by Jelmer Vernooij.
• sane-backends/1.0.24-11 by J rg Frings-F rst timestamps in .dvi and .ps
• tomcat6/6.0.41-4 by Emmanuel Bourg.
• tomcat7/7.0.61-1 by Emmanuel Bourg; currently FTBFS.
• tomcat8/8.0.22-2 by Emmanuel Bourg.

Patches submitted which did not make their way to the archive yet:

• #784541 on yasm by Lunar: remove build date from version strings.
• #784694 on smcroute by Micha Lenk: remove build date from version string.
• #784672 on gnumeric by Daniel Kahn Gillmor: remove timestamps in embedded gzip'ed data in shared library.
• #774347 on sed by Lunar: fix permissions before creating the package.
• #784352 on icebreaker by Reiner Herrmann: use UTC timezone when calculating version date.
• #784325 on kde-workspace by Lunar: make the output of kdm confproc.pl stable.
• #784602 on monkeysign by Daniel Kahn Gillmor: use time of debian/changelog entry when generating documentation.
• #784723 on alot by Juan Picca: pass time of debian/changelog entry to Sphinx.
• #784538 on file-rc by Lunar: use sed instead of grep+mv to keep correct file permissions.
• #784335 on libapache2-mod-perl2 by Lunar: set PERL_HASH_SEED=0 during configure to make the generated .c and .h files stable.
• #784267 on mpv by Lunar: pass --disable-build-date to ./configure.
• #784793 on bugs-everywhere by Daniel Kahn Gillmor: use time of debian/changelog entry as build date.
• #784318 on gnome-desktop3 by Lunar: use time of debian/chanelog entry as build date.
• #774504 on debianutils by Lunar: fix file permissions.

reproducible.debian.net Alioth now hosts a script that can be used to redo builds and test for a package. This was preliminary done manually through requests over the IRC channel. This should reduce the number of interruptions for jenkins' maintainers The graph of the oldest build per day has been fixed. Maintainance scripts will not error out when they are no files to remove. Holger Levsen started work on being able to test variations of CPU features and build date (as in build in another month of 1984) by using virtual machines. debbindiff development Version 18 has been released. It will uses proper comparators for pk3 and info files. Tar member names are now assumed to be UTF-8 encoded. The limit for the maximum number of different lines has been removed. Let's see on reproducible.debian.net how it goes for pathological cases. It's now possible to specify both --html and --text output. When neither of them is specified, the default will be to print a text report on the standard output (thanks to Paul Wise for the suggestion). Documentation update Nicolas Boulenguez investigated Ada libraries. Package reviews 451 obsolete reviews have been removed and 156 added this week. New identified issues: running kernel version getting captured, random filenames in GHC debug symbols, and timestamps in headers generated by qdbusxml2cpp. Misc. Holger Levsen went to re:publica and talked about reproducible builds to developers and users there. Holger also had a chance to meet FreeBSD developers and discuss the status of FreeBSD. Investigations have started on how it could be made part of our current test system. Laurent Guerby gave Lunar access to systems in the GCC Compile Farm. Hopefully access to these powerful machines will help to fix packages for GCC, Iceweasel, and similar packages requiring long build times.

Debian Jessie has been released on April 25th, 2015. This has opened the Stretch development cycle. Reactions to the idea of making Debian build reproducibly have been pretty enthusiastic. As the pace is now likely to be even faster, let's see if we can keep everyone up-to-date on the developments. Before the release of Jessie The story goes back a long way but a formal announcement to the project has only been sent in February 2015. Since then, too much work has happened to make a complete report, but to give some highlights:

• New variations are now tested: umask, kernel version, domain name, and timezone. We might only be missing CPU type and current date now.
• Many improvements to the test system on jenkins.debian.net and the pages showing the results.
• Now not only packages from unstable are tested but also those in testing and experimental.
• When rescheduling packages for testing, the build products can be kept and the IRC channel gets a notification when its over.
• binutils version 2.25-6 is now built with the --enable-deterministic-archives flag. Making ar, strip and others create deterministic static libraries.
• Number of identified issues has grown from about 80 to 123 today.

Lunar did a pretty improvised lightning talk during the Mini-DebConf in Lyon. This past week It seems changes were pilling behind the curtains given the amount of activity that happened in just one week. Toolchain fixes

• Niels Thykier uploaded debhelper/9.20150501 which includes fixes to dh_makeshlibs (#774100), dh_icons (#774102), dh_usrlocal (#775020). Patches written by Lunar.
• Helmut Grohne uploaded doxygen/1.8.9.1-3 which will not generate timestamps in HTML by default. Kudos to akira for bringing the issue upstream.
• Kenneth J. Pronovici uploaded epydoc/3.0.1+dfsg-6 adding a --no-include-build-time option. Patch by Jelmer Vernooij.
• David Pr vot uploaded php-apigen/2.8.1+dfsg-2 which now has reproducible output.
• C dric Boutillier uploaded ruby-prawn/2.0.1+dfsg-1 which now produce a deterministic output when using gradients. Patch by Lunar.
• Jelmer Vernooij uploaded samba/2:4.1.17+dfsg-4 which contains a patch by Matthieu Patou making the output of pidl (from libparse-pidl-perl) reproducible.
• Dmitry Shachnev uploaded sphinx/1.3.1-1 in experimental which should produce deterministic output. The original patch from Chris Lamb has inspired the upstream fix.
• gregor herrmann uploaded libextutils-depends-perl/0.404-1 which makes ExtUtils::Depends output deterministic. Original patch by Reiner Herrmann.
• Niko Tyni uploaded perl/5.20.2-4 which makes the output of Pod::Man reproducible. Nice team work visible on #780259.

We also rebased the experimental version of debhelper twice to merge the latest set of changes. Lunar submitted a patch to add a -creation-date to genisoimage. Reiner Herrmann opened #783938 to request making -notimestamp the default behavior for javadoc. Juan Picca submitted a patch to add a --use-date flag to texi2html. Packages fixed The following packages became reproducible due to changes of their build dependencies: apport, batctl, cil, commons-math3, devscripts, disruptor, ehcache, ftphs, gtk2hs-buildtools, haskell-abstract-deque, haskell-abstract-par, haskell-acid-state, haskell-adjunctions, haskell-aeson, haskell-aeson-pretty, haskell-alut, haskell-ansi-terminal, haskell-async, haskell-attoparsec, haskell-augeas, haskell-auto-update, haskell-binary-conduit, haskell-hscurses, jsch, ledgersmb, libapache2-mod-auth-mellon, libarchive-tar-wrapper-perl, libbusiness-onlinepayment-payflowpro-perl, libcapture-tiny-perl, libchi-perl, libcommons-codec-java, libconfig-model-itself-perl, libconfig-model-tester-perl, libcpan-perl-releases-perl, libcrypt-unixcrypt-perl, libdatetime-timezone-perl, libdbd-firebird-perl, libdbix-class-resultset-recursiveupdate-perl, libdbix-profile-perl, libdevel-cover-perl, libdevel-ptkdb-perl, libfile-tail-perl, libfinance-quote-perl, libformat-human-bytes-perl, libgtk2-perl, libhibernate-validator-java, libimage-exiftool-perl, libjson-perl, liblinux-prctl-perl, liblog-any-perl, libmail-imapclient-perl, libmocked-perl, libmodule-build-xsutil-perl, libmodule-extractuse-perl, libmodule-signature-perl, libmoosex-simpleconfig-perl, libmoox-handlesvia-perl, libnet-frame-layer-ipv6-perl, libnet-openssh-perl, libnumber-format-perl, libobject-id-perl, libpackage-pkg-perl, libpdf-fdf-simple-perl, libpod-webserver-perl, libpoe-component-pubsub-perl, libregexp-grammars-perl, libreply-perl, libscalar-defer-perl, libsereal-encoder-perl, libspreadsheet-read-perl, libspring-java, libsql-abstract-more-perl, libsvn-class-perl, libtemplate-plugin-gravatar-perl, libterm-progressbar-perl, libterm-shellui-perl, libtest-dir-perl, libtest-log4perl-perl, libtext-context-eitherside-perl, libtime-warp-perl, libtree-simple-perl, libwww-shorten-simple-perl, libwx-perl-processstream-perl, libxml-filter-xslt-perl, libxml-writer-string-perl, libyaml-tiny-perl, mupen64plus-core, nmap, openssl, pkg-perl-tools, quodlibet, r-cran-rjags, r-cran-rjson, r-cran-sn, r-cran-statmod, ruby-nokogiri, sezpoz, skksearch, slurm-llnl, stellarium. The following packages became reproducible after getting fixed:

• berkeley-abc/1.01+20141105hg5b5af75+dfsg-3 uploaded by Ruben Undheim, original patch by Johann Klammer.
• binutils/2.25-7 by Matthias Klose.
• bitmap-mule/8.5+0.20030825.0433-16 uploaded by Tatsuya Kinoshita, original patch by Chris Lamb.
• blobby/1.0-2 by Felix Geyer.
• codelite/7.0+dfsg-2 by James Cowgill.
• conkeror/1.0~~pre-1+git150409-1 by Axel Beckert.
• convmv/1.15-1 by Christian Perrier.
• cpl/6.6~b-1 by Ole Streicher.
• deheader/1.1-2 by Reiner Herrmann.
• dict-foldoc/20150318-1 uploaded by Iustin Pop, original patch by Chris Lamb.
• ding/1.8-1 by Roland Rosenfeld.
• doc-rfc/20150425-1 by Iustin Pop.
• doxygen/1.8.9.1-3 by Helmut Grohne.
• eureka/1.07-1 by Fabian Greffrath.
• eximdoc4/4.85-2 uploaded by Andreas Metzler, original patch by Chris Lamb.
• flashproxy/1.7-3 by Ximin Luo.
• heimdal/1.6~rc2+dfsg-10 by Jelmer Vernooij.
• init-system-helpers/1.23 by Martin Pitt original patch by Lunar.
• ldb/2:1.1.20-2 by Jelmer Vernooij.
• libalien-wxwidgets-perl/0.67+dfsg-1 uploaded by gregor herrmann, original patch by Chris Lamb.
• libcairo-perl/1.105-1 by intrigeri.
• libclass-methodmaker-perl/2.24-1 uploaded by gregor herrmann, original patch by Chris Lamb.
• libcommon-sense-perl/3.73-3 uploaded by gregor herrmann, original patch by Chris Lamb.
• libelixirfm-perl/1.1.976-4 uploaded by gregor herrmann, original patch by Chris Lamb.
• libgnome2-perl/1.045-3 by intrigeri.
• libjs-jcrop/0.9.12+dfsg-2 uploaded by David Pr vot, original patch by Chris Lamb.
• liblas/1.8.0-2 by Bas Couwenberg.
• libopengl-perl/0.6704+dfsg-1 uploaded by gregor herrmann, original patch by Chris Lamb.
• libparse-recdescent-perl/1.967009+dfsg-2 uploaded by gregor herrmann, original patch by Reiner Herrmann.
• librasterlite/1.1g-5 by Bas Couwenberg.
• libterm-size-perl-perl/0.029-2 uploaded by gregor herrmann, original patch by Chris Lamb.
• libvdpau/1.1-1 by Andreas Beckmann.
• libxml-sax-expatxs-perl/1.33-2 uploaded by gregor herrmann, original patch by Chris Lamb.
• lynx-cur/2.8.9dev4-2 by Axel Beckert.
• mapcache/1.2.1-3 by Bas Couwenberg.
• mdbtools/0.7.1-4 by Jean-Michel Nirgal Vourg re.
• mopidy uploaded by Stein Magnus Jodal, fixed by upstream.
• objenesis/2.1-1 by Markus Koschany.
• opendkim/2.10.1-2 uploaded by Scott Kitterman, original patch by Reiner Herrmann.
• pktools/2.6.3-1 by Bas Couwenberg.
• posh/0.12.4 uploaded by Clint Adams, original patch by Chris Lamb.
• prboom-plus/2:2.5.1.4~svn4425+dfsg1-1 by Fabian Greffrath.
• qlandkartegt/1.8.1+ds-1 by Bas Couwenberg.
• readosm/1.0.0d-1~exp2 by Bas Couwenberg.
• robocode/1.9.2.4-1 by Markus Koschany.
• ruby-prawn/2.0.1+dfsg-1 uploaded by C dric Boutillier, original patch by Lunar.
• sane-backends/1.0.25+git20150425-1 by J rg Frings-F rst.
• scoop/0.7.1-3 by Daniel Stender.
• springlobby/0.218-1 by Markus Koschany.
• subvertpy/0.9.2-1 by Jelmer Vernooij.
• t-prot/3.4-2 by Axel Beckert.
• talloc/2.1.2-3 by Jelmer Vernooij.
• tdb/1.3.4-2 by Jelmer Vernooij, patch now applied by upstream.
• tk-html3/3.0~fossil20110109-5 by Ole Streicher.
• tox/1.9.2-2 uploaded by Barry Warsaw original patch by Reiner Herrmann.
• trove3/3.0.3-2 by Erich Schubert.
• txt2man/1.5.6-2 uploaded by Joao Eriberto Mota Filho, initial patch by Jonathan Wiltshire.
• units/2.11-2 by Stephen Kitt.
• win32-loader/0.7.10 upload by Didier Raboud, original patch by Lunar.
• zec/0.12-3 uploaded by Clint Adams, original patch by Chris Lamb.
• zomg/0.8-1 uploaded by Clint Adams, original patch by Chris Lamb.

Some uploads fixed some reproducibility issues but not all of them:

• ada-reference-manual/1:2012.2-4 by Nicolas Boulenguez.
• adacontrol/1.16r11-3 by Nicolas Boulenguez.
• aspectj/1.8.5-1 by Emmanuel Bourg.
• binutils-m68hc1x/1:2.18-4 uploaded by Riku Voipio, original patch by Chris Lamb.
• debianutils/4.5) uploaded by Clint Adams, original patch by Lunar.
• ioquake3/1.36+u20150412+dfsg1-1 by Simon McVittie.
• libsdl1.2/1.2.15-11 uploaded by Manuel A. Fernandez Montecelo, original patch by Chris Lamb; currently FTBFS.
• libsdl2/2.0.2+dfsg1-7 by Manuel A. Fernandez Montecelo, original patch by Chris Lamb; currently FTBFS.
• nedit/1:5.6a-2 by Paul Gevers.
• openarena-085-data/0.8.5split-4 by Simon McVittie.
• openarena-data/0.8.5split-4 by Simon McVittie.
• openarena/0.8.8-12 by Simon McVittie.
• openchange/1:2.2-6 by Jelmer Vernooij.
• puredata/0.46.6-1 by IOhannes m zm lnig.
• pyexiv2/0.3.2-8 by Michal iha ; currently FTBFS.
• quakespasm/0.90.0-2 by Stephen Kitt.
• sed/4.2.2-5 by Clint Adams, patch by Lunar.
• v4l2loopback/0.8.0-5 uploaded by IOhannes m zm lnig, original patch by Chris Lamb.
• vim/2:7.4.712-1 uploaded by James McCoy, original patch by Reiner Herrmann.
• zookeeper/3.4.6-4 by Emmanuel Bourg.

Patches submitted which did not make their way to the archive yet:

• #783882 on jodconverter by Reiner Herrmann: tell javadoc to stop writing timestamps.
• #783885 on bind9 by Reiner Herrmann: pass -DNO_VERSION_DATE to build system.
• #783688 on serverstats by Chris Lamb: fix permissions.
• #783453 on cdbackup by Chris Lamb: remove build date from version string in binary.
• #783478 on texi2html by Juan Picca: pass --use-date to texi2html.
• #783933 on imagemagick by Reiner Herrmann: remove timestamps from PNG and use deterministic package build date.
• #783515 on memtest86+ by Lunar: fix embedded mtimes and pass -creation-date to genisoimage.
• #783558 on websvn by Chris Lamb: fix permissions.

Improvements to reproducible.debian.net Mattia Rizzolo has been working on compressing logs using gzip to save disk space. The web server would uncompress them on-the-fly for clients which does not accept gzip content. Mattia Rizzolo worked on a new page listing various breakage: missing or bad debbindiff output, missing build logs, unavailable build dependencies. Holger Levsen added a new execution environment to run debbindiff using dependencies from testing. This is required for packages built with GHC as the compiler only understands interfaces built by the same version. debbindiff development Version 17 has been uploaded to unstable. It now supports comparing ISO9660 images, dictzip files and should compare identical files much faster. Documentation update Various small updates and fixes to the pages about PDF produced by LaTeX, DVI produced by LaTeX, static libraries, Javadoc, PE binaries, and Epydoc. Package reviews Known issues have been tagged when known to be deterministic as some might unfortunately not show up on every single build. For example, two new issues have been identified by building with one timezone in April and one in May. RD and help2man add current month and year to the documentation they are producing. 1162 packages have been removed and 774 have been added in the past week. Most of them are the work of proper automated investigation done by Chris West. Summer of code Finally, we learned that both akira and Dhole were accepted for this Google Summer of Code. Let's welcome them! They have until May 25th before coding officialy begins. Now is the good time to help them feel more comfortable by sharing all these little bits of knowledge on how Debian works.

For a while, I've been wanting to set up configuration management for my home network. With half a dozen servers, a VPS and a workstation it is not big, but large enough to make it annoying to manually log into each machine for network-wide changes. Most of the servers I have are low-end ARM machines, each responsible for a couple of tasks. Most of my machines run Debian or something derived from Debian. Oh, and I'm a member of the declarative school of configuration management.

A whopping 5 years ago, LWN ran a story about distributed bug trackers. This was during the early waves of distributed version control adoption, and so everybody was looking for other things that could benefit from decentralization. TL;DR: Not much has changed since. The potential benefits of a distributed bug tracker are similar to those of a distributed version control system: ability to fork any arbitrary project, easier collaboration between related projects and offline access to full project data. The article discussed a number of systems, including Bugs Everywhere, ScmBug, DisTract, DITrack, ticgit and ditz. The conclusion of our favorite grumpy editor at the time was that all of the available distributed bug trackers were still in their infancy. All of these piggyback on a version control system somehow - either by reusing the VCS database, by storing their data along with the source code in the tree, or by adding custom hooks that communicate with a central server. Only ScmBug had been somewhat widely deployed at the time, but its homepage gives me a blank page now. Of the trackers reviewed by LWN, Bugs Everywhere is the only one that is still around and somewhat active today. In the years since the article, a handful of new trackers have come along. Two new version control systems - Veracity and Fossil - come with the kitchen sink included and so feature a built-in bug tracker and wiki. There is an extension for Mercurial called Artemis that stores issues in an .issues directory that is colocated with the Mercurial repository. The other new tracker that I could find (though it has also not changed since 2009) is SD. It uses its own distributed database technology for storing bug data - called Prophet, and doesn't rely on a VCS. One of the nice features is that it supports importing bugs from foreign trackers. Some of these provide the benefits you would expect of a distributed bug tracker. Unfortunately, all those I've looked at fail to even provide the basic functionality I would want in a bug tracker. Moreso than with a version control system, regular users interact with a bug tracker. They report bugs, provide comments and feedback on fixes. All of the systems I tried make these actions a lot harder than with your average bugzilla or mantis instance - they provide a limited web UI or no web interface at all. Update: LWN later also publishe articles on on SD and on Fossil. Other interesting links are Eric Sink's article on distributed bug tracking (Erik works at Sourcegear who develop Veracity) and the dist-bugs mailing list.

Dear lazyweb, I've been reading about what the rest of the world seems to be calling "quantified self". In essence, it is tracking of personal data like activity, usually with the goal of data-driven decision making. Or to take a less abstract common example: counting the number of steps you take each day to motivate yourself to take more. I wish it'd been given a less annoying woolly name but this one seems to have stuck. There are a couple of interesting devices available that track sleep, activity and overall health. Probably best known are the FitBit and the jazzed-up armband pedometers like the Jawbone UP and the Nike Fuelband. Unfortunately all existing devices seem to integrate with cloud services somehow, rather than giving the user direct access to their data. Apart from the usual privacy concerns, this means that it is hard to do your own data crunching or create a dashboard that contains data from multiple sources. Has anybody found any devices that don't integrate with the cloud and just provide raw data access?

"porcelain" is the term that is usually used in the Git world to refer to the user-facing parts. This is opposed to the lower layers: the plumbing. For a long time, I have resisted the idea of including a porcelain layer in Dulwich. The main reason for this is that I don't consider Dulwich a full reimplementation of Git in Python. Rather, it's a library that Python tools can use to interact with local or remote Git repositories, without any extra dependencies. dulwich has always shipped a 'dulwich' binary, but that's never been more than a basic test tool - never a proper tool for end users. It was a mistake to install it by default. I don't think there's a point in providing a dulwich command-line tool that has the same behaviour as the C Git binary. It would just be slower and less mature. I haven't come across any situation where it didn't make sense to just directly use the plumbing. However, Python programmers using Dulwich seem to think of Git operations in terms of porcelain rather than plumbing. Several convenience wrappers for Dulwich have sprung up, but none of them is very complete. So rather than relying on external modules, I've added a "porcelain" module to Dulwich in the porcelain branch, which provides a porcelain-like Python API for Git. At the moment, it just implements a handful of commands but that should improve over the next few releases:

from dulwich import porcelain
r = porcelain.init("/path/to/repo")
porcelain.commit(r, "Create a commit")
porcelain.log(r)

Packt recently published a book on Version Control using Bazaar written by Janos Gyerik. I was curious what the book was like, and they kindly provided me with a digital copy. The book is split into roughly five sections: an introduction to version control using Bazaar's main commands, an overview of the available workflows, some chapters on the available extensions and integration, some more advanced topics and finally, a quick introduction to programming using bzrlib. It is assumed the reader has no pre-existing knowledge about version control systems. The first chapters introduce the reader to the concept of revision history, branching and merging and finally collaboration. All concepts are first discussed in theory, and then demonstrated using the Bazaar command-line UI and the bzr-explorer tool. The book follows roughly the same track as the official documentation, but it is more extensive and has more fancy drawings of revision graphs. The middle section of the book discusses the modes in which Bazaar can be used - centralized or decentralized - as well as the various ways in which code can be landed in the main branch ("workflows"). The selection of workflows in the book is roughly the same as those in the official Bazaar documentation. The author briefly touches on a number of other software engineering topics such as code reviews, code formatting and automated testing, though not sufficiently to make it useful for people who are unfamiliar with these techniques. Both the official documentation and the book complicate things unnecessarily by listing every possible option. The next chapter is a basic howto on the use of Bazaar with various hosting solutions, such as Launchpad, Redmine and Trac. The Advanced Features chapter covers a wide range of obscure and less obscure features in Bazaar: uncommit, shelves, re-using working trees, lightweight checkouts, stacked branches, signing revisions and using e-mail hooks. The chapter on foreign version control system integration is a more extensive version of the public docs. It has some factual inaccuracies; in particular, it recommends the installation of a 2 year old buggy version of bzr-git. The last chapter provides quite a good introduction to the Bazaar APIs and plugin writing. It is a fair bit better than what is available publically. Overall, it's not a bad book but also not a huge step forward from the official documentation. I might recommend it to people who are interested in learning Bazaar and who do not have any experience with version control yet. Those who are already familiar with Bazaar or another version control system will not find much new. The book misses an opportunity by following the official documentation so closely. It has the same omissions and the same overemphasis on describing every possible feature. I had hoped to read more about Bazaar's data model, its file format and some of the common problems, such as parallel imports, format hell and slowness.

A while ago I migrated most of my packages from Bazaar to Git. The rest of the world has decided to use Git for version control, and I don't have enough reason to stubbornly stick with Bazaar and make it harder for myself to collaborate with others. So I'm moving away from a workflow I know and have polished over the last few years - including the various bzr plugins and other tools involved. Trying to do the same thing using git is frustrating and time-consuming, but I'm sure that will improve with time. In particular, I haven't found a good way to merge in a new upstream release (from a tarball) while referencing the relevant upstream commits, like bzr merge-upstream can. Is there a good way to do this? What helper tools can you recommend for maintaining a Debian package in git? Having been upstream for bzr-git earlier, I used its git-remote-bzr implementation to do the conversions of the commits and tags:

% git clone bzr::/path/to/bzr/foo.bzr /path/to/git/foo.git

One of my last contributions to bzr-git was a bzr git-push-pristine-tar-deltas subcommand, which will export all bzr-builddeb-style pristine-tar metadata to a pristine-tar branch in a Git repository that can be used by pristine-tar directly or through something like git-buildpackage. Once you have created a git clone of your bzr branch, it should be a matter of running bzr git-push-pristine-tar-deltas with the target git repository and the Debian package name:

% cd /path/to/bzr/foo.bzr
% bzr git-push-pristine-tar-deltas /path/to/git/foo.git foo
% cd /path/to/git/foo.git foo
% git branch
*  master
   pristine-tar

Apparently 'tis the season for major software releases. Julien has just announced the release of OpenChange 2.0, codenamed quadrant. This release fixes a number of important bugs and enables integration with SOGo. With the SOGo backend, it is now possible to set up an Exchange-compatible groupware server that can be accessed from Outlook without the need to connect any connectors. See the release notes for more details.

For the last 7 years I've been involved in the Bazaar project. Since I am slowly stepping down, I recently wrote a retrospective on the project as I experienced it for the last 7 years. Thanks to a few kind people for proofreading earlier drafts; if you spot any errors, please let me know in the comments.

This afternoon we released version 4.0.0 of Samba. This is a significant milestone, and I'm very proud of the result. Samba 4 is the first version that can be a domain controller in an Active Directory domain. We embarked on this journey almost a decade ago - the first commit is from August 2003. It's been a long and bumpy ride. I hardly recognize the people in this team photo from 2003 (I'm second from the left). A lot has happened in that time. We wrote a few million lines of code. We migrated from CVS to Subversion to Git. We've drifted apart and grown back together as a team. In my youthful naivity I predicted a release "within 1 or 2 years" during a talk at the NLUUG in 2004. But Active Directory was a lot harder than we thought, and there were quite a few other distractions as well. I'm glad this release, which is by far the biggest and longest running software project I have ever worked on, has finally happened. Some older RCs of Samba 4 have already been packaged for Debian and Ubuntu, in the samba4 source package. For Debian jessie, these will be integrated into the main samba source package. Please use experimental if you do want to try the existing packages, as it is most up to date.

A new release 0.9.8 of Rcpp is now on CRAN and will also get into Debian shortly (once I finish building R 2.14.1). This release contains a few incremental changes. Romain, sponsored by by the Open Source Programs Office at Google, had released a new package int64 bringing larger integers to R, and this is now supported by Rcpp as well. John Chambers contributed some code to have Reference Classes extend existing C++ classes (typically brought in via Rcpp Modules). Jelmer Ypma sent us a patch to add a Rcout device not unlike cout, but aligned with R's io buffering. We added some more unit tests, and made a few small fixes here or there. The complete NEWS entry is below; more details are in the ChangeLog file in the package and on the Rcpp Changelog page.

0.9.8   2011-12-21
    o   wrap now handles 64 bit integers (int64_t, uint64_t) and containers 
        of them, and Rcpp now depends on the int64 package (also on CRAN).
        This work has been sponsored by the Google Open Source Programs
        Office.
    o   Added setRcppClass() function to create extended reference classes 
        with an interface to a C++ class (typically via Rcpp Module) which
        can have R-based fields and methods in addition to those from the C++.
    o   Applied patch by Jelmer Ypma which adds an output stream class
        'Rcout' not unlike std::cout, but implemented via Rprintf to
        cooperate with R and its output buffering.
        
    o   New unit tests for pf(), pnf(), pchisq(), pnchisq() and pcauchy()
    o   XPtr constructor now checks for corresponding type in SEXP
    o   Updated vignettes for use with updated highlight package
    o   Update linking command for older fastLm() example using external 
        Armadillo

Thanks to CRANberries, you can also look at a diff to the previous release 0.9.7. As always, even fuller details are on the Rcpp Changelog page and the Rcpp page which also leads to the downloads, the browseable doxygen docs and zip files of doxygen output for the standard formats. A local directory has source and documentation too. Questions, comments etc should go to the rcpp-devel mailing list off the R-Forge page

I attended the Git Together earlier this week. I was tenative about this, since I'm not really much of a git developer; all my git work is building stuff on top of it. It turned out great though. At first it seemed like one of those parties where you don't know anyone. But then I got to reconnect with Avery Pennarun for the first time since DebConf 2, and got to know Jonathan Nieder better, and it was also nice to see Jelmer Vernooij. And the core developers were also very welcoming. Junio Hamano knew of my work (and I am in awe of his), and Jeff King thinks my take on SHA1 security issues has value, and has been expanding on it. Shawn Pearce managed the unconference subtly and well. Lots of very smart people. At one point I found myself accross the table from Android's lead developer. I was very happy that everything I think needs improvement in git was discussed during the unconference:

• big files: My postit suggesting this got more checks than most anything else, and I briefly presented git-annex at the start of a session on general scalability -- on its 1-year anniversary. Some ideas for improved hooks that git-annex and other tools could use are developing. Better scalability to lots of files and more efficient index files were also discussed.
• git as a filesystem: There was a consensus that gone are the days when git was just about managing source code. (I remember being told on #git before I wrote etckeeper, that no, git should not be used for that..)
• submodules: I was astounded that they're now considering supporting "floating" submodules, which would track the head of a branch, rather than the specific rev committed in the superproject. Many other problems that have kept me from ever trying submodules are also being worked on. This seems unlikely to replace mr, but who knows -- at least getting rid of repo is a goal.
• SHA1 security was discussed for quite a long while, long enough that I felt a bit guilty for bringing it up, but it was an interesting and fruitful discussion. I went in thinking that the checksum basically has to be parameterized, but they have some good reasons not to do that, and some other good ideas, although what to do and when best to do it is still open for discussion. Signed commits are certianly coming soon. Also this amazing patch was developed.
• Metadata storage was briefly discussed, but nobody seemed sure how to deal with it. Ideas floated included a metastore like tool that uses mergeable files, or storing metadata in some sort of notes-like separate branch.

The Samba team is once again participating in the Summer of Code this year. This year we have 4 students working on various projects related to Samba. This year I am mentoring Dhananjay Sathe, who is improving the GTK+ frontends for Samba. In particular, he is making it possible to manage shares and users of a remote Samba or Windows machine. Dhananjay is also blogging about his progress.

It's been more than a month since the last status update on my Samba 4 work - much more than the two weeks I promised. During the holidays I finally managed to release the new alpha of Samba 4, as well as releases of some of our companion libraries (tdb, talloc, tevent and ldb). The release includes a significant amount of bug fixes and a lot of work towards a properly functioning Active Directory DC, too much to list here. This release I've mainly been involved in improving our Python bindings and our handling of internal and external libraries. We now use symbol versioning for our copy of Heimdal Kerberos as well as some of our other libraries. Hopefully this will fix some of the issues users of the evolution-mapi plugin have been seeing where they end up with both MIT Kerberos and Heimdal Kerberos loaded into the same process (with all the consequences of overlapping symbol names). Samba 4 now also has the ability to work with the system Heimdal rather than using the bundled copy. I have packaged alpha14 for Debian and Ubuntu (fixing most of the open bugs against the Samba 4 package in the BTS), but am currently waiting for the new release of ldb to pass through NEW before I can upload. The next release is scheduled for the first week of February. cp: Stream of Passion - Haunted

After Samba XP 2008 Andrew and I started keeping a wiki page with our bi-weekly goals and achievements for Samba 4. Because planning in a Free Software project is hard (time availability and priorities change over time, and other volunteers are equally unpredictable) we called this our "Fantasy page"; it listed things we wanted to work on next ("fantasies"), but reality being what it is we would usually actually end up working on something entirely different. We discussed our progress and new plans in - what I would now call - a bi-weekly standup call. There were several reasons for doing this. It gave us some sense of direction as well as a sense of accomplishment; a way to look back at the end of the year and realize how much we had actually achieved. Because Samba 4 is such a long term project (it is 7 years old at this point) it is easy to become disillusioned, to look back at a year of commits and to not see the gradual improvement, just the fact that there is no release yet. We managed to keep this up for two years, much longer than I had anticipated, and eventually started to slip last year. More recently Kai and Tridge have started to blog weekly about their efforts to make Samba 4.0 a reality and I'm going to join them by trying to blog regularly - every two weeks - about my contributions, even if there were none. In the next two weeks I plan to work on finally getting alpha 14 of Samba 4 out and on fixing the daily builds of Samba 4 and OpenChange for Ubuntu on Launchpad after we did a massive reorganization of the private libraries in Samba 4. cp: Zero 7 - Somersault

It s already the fourth issue of Flattr FOSS: it means 20 different projects using Flattr that I presented you. Here are the 5 suggestions for November:

1. Redshift is a small utility that adjusts the color temperature of your screen to make it less aggressive on your eyes, in particular during evening/nights. It uses the time of the day and the geographic location to know whether it s night or day. I discovered it this summer and I liked it, although I m not running it permanently.
2. Noscript is a Firefox plugin to control what sites can execute javascript, flash and other plugins. All those are creating supplementary security risks and you browse safer if you allow only some sites to run them. This is the number one entry on Flattr in the opensource category, it recently took the place of dpkg.
3. phyMyAdmin is a web interface to manage MySQL databases. If you have such a database on a web-hosting service, you have likely already seen it in action. It s an award-winning software with a 12 year history, it s not so common for PHP applications.
4. Chromium maintenance in Debian. Chromium is a rapidly-evolving & complex software and Giuseppe Iuculano has been tirelessly working on packaging it. Almost alone within Debian. He deserves kudos for his work even though he reused work made by Fabien Tassin on the Ubuntu package.
5. Dulwich is a pure-Python implementation of the Git file formats and protocols. It s an important building block for interoperability between Bazaar and Git: bzr-git (a Bazaar plugin providing Git integration) is notably using it. Given the large usage of bzr in Ubuntu and the popularity of Git world-wide, it s important to have such gateways.

This article is part of the Flattr FOSS project.

Update: mentioned the work of Fabien Tassin on the chromium package within Ubuntu.

6 comments Support my work

There's more good news on the OpenChange front. Julien has been working together with Wolfgang and Ludovic from Inverse recently to leverage the server-side support in OpenChange to provide native Exchange server support in SOGo. A couple of days ago we announced that there now is an initial version that allows the use of Outlook against a SOGo server through OpenChange.

<object data="http://www.youtube.com/v/oSZJ95YeXYE?fs=1&amp;hl=en_US&amp;hd=1" height="385" type="application/x-shockwave-flash" width="480"><param name="movie" value="http://www.youtube.com/v/oSZJ95YeXYE?fs=1&amp;hl=en_US&amp;hd=1"><param name="movie" value="http://www.youtube.com/v/oSZJ95YeXYE?fs=1&amp;hl=en_US&amp;hd=1"><param name="allowFullScreen" value="true"><param name="allowscriptaccess" value="always"></object> (alternatively you can find the screencast here) As far as I know, this is the first time it's possible to actually use Outlook clients with a non-Microsoft Exchange-compatible server without the need for plugins on the Outlook side. And it's all Free Software. Of course, this is just a preview, and not something we'd recommend everybody to run in production yet. But it's exciting to finally see this come together. We already have OpenChange packages in Debian and Ubuntu but I hope I can help get SOGo packaged for both distributions as well.

Daily builds As of a month ago there are Ubuntu archives with fresh packages of Samba 4 and Openchange, built on a daily basis day from the latest upstream revision. This means that it is now possible to run a version of Samba 4 that is less than 24 hours old, without having to know how to extract source code from the version control system that upstream is using, without having to know how to build and install an application from source, but perhaps most importantly: without having to go through the tedious process of manually updating the source code and rebuilding. OpenChange is tightly coupled to Samba 4, so installing a new version of OpenChange usually involves installing a new version of Samba 4 as well. To make matters more confusing, the two projects use different version control systems (Samba 4 is in Git, while OpenChange is in Subversion) and different build systems (Samba 4 uses waf, OpenChange uses autoconf and make). I have been involved in Samba 4 and OpenChange as an upstream developer and more recently also as a packager for both Debian and Ubuntu. As an upstream developer for both these projects it is important for me that users can easily run the development versions. It makes it possible for interested users to confirm the fixes for issues they have reported and to test new features. The more users run the development version, the more confident I can be as a developer that doing a release will not cause any unexpected surprises. As a packager it is useful to know when there are upstream changes that are going to break my package with the next release.

Recipes The daily builds work using so-called recipes which describe how to build a Debian source package from a set of Bazaar branches. For example, the Samba 4 recipe looks like this:

# bzr-builder format 0.2 deb-version 4.0.0~alpha14~bzr revno ~ppa revno:packaging + revno:debian 
lp:samba
merge debian lp:~samba-team/samba/unstable
merge packaging lp:~samba-team/samba/4.0-ppa-maverick

This dictates that a source package should be built by taking the upstream Samba branch and merging the Debian packaging and some recipe-specific tweaking. The last bit on the first line indicates the version string to be used when generating a changelog entry for the daily build. Every night Launchpad (through bzr-builder) merges these branches and attempts to build the resulting source package, e-mailing me in case of build problems. Generally I fix issues that come up by committing directly to upstream VCS or to the Debian packaging branch. There is no overhead in maintaining the daily build after I've set it up. For more information on creating source package recipes, see getting started.

Toolchain The entire toolchain that does the daily package builds for Ubuntu is Free Software, and I have contributed to various bits of that toolchain over the years. It's exciting to see everything come together.

Soyuz Launchpad consists of multiple pillars - one of those pillars is Soyuz, which I hack on as part of my day job at Canonical. Soyuz is responsible for the archive management and package building. Debian source packages (a combination of upstream source code and packaging metadata) get uploaded by users and then built for various architectures on our buildfarm and published to the Ubuntu archive or to users personal package archives.

Launchpad-code Another pillar of Launchpad is Launchpad-code, which is responsible for the hosting and management of version control branches. Launchpad users can either host their branches on Launchpad directly or mirror branches (either native Bazaar branches or branches in a foreign format such as Subversion, Git or Mercurial). The mirrorring of native and foreign branches happens using standard Bazaar API's. In the case of Samba and OpenChange we import the branches of the upstream projects (Samba is in Git, OpenChange is in Subversion) and the packaging for both projects is in Bazaar. Launchad-code calls out to Bazaar to do the actual mirrorring. Over the last few years I have done a lot of work to improve Bazaars support for foreign branches, in particular on supporting Subversion, Git and Mercurial. As the code mirrorring in Launchpad is one of the biggest users of bzr-svn and bzr-git it has helped find some of the more obscure bugs in those plugins over the last few years, to the point where there are only a handful of issues with Git and Subversion imports left.

bzr-git and dulwich bzr-git provides transparent access to Git repositories from within Bazaar and is built on top of Dulwich. Dulwich is a Python library that provides access to the Git file formats and protocols that is completely independent of Bazaar. James Westby originally started it and I adopted it for bzr-git and further extended it. There are now several other projects that use it as well, including hg-git, and rabbitvcs. Apart from James and me almost two dozen other people have contributed it so far.

bzr-svn and subvertpy bzr-svn provides transparant access to Subversion repositories in Bazaar. When I grew frustrated with the existing Subversion Python bindings for various reasons, I decided to create independent Python bindings for Subversion from scratch. These bindings have since been split out into a separate project - subvertpy - and other projects have since also started using them, e.g. hgsubversion and basie.

Using the daily builds To use the Samba 4 and OpenChange daily builds (Ubuntu Maverick only for now), run:

$ apt-add-repository ppa:samba-team/ppa
$ apt-add-repository ppa:openchange/daily-builds

cp: Karnivool - Themata

Seeing this makes me very happy. It's taken us a couple of years to get to this point but we've finally made it, mostly thanks to the dedication and persistence of Julien and Brad.